Papers with sparsity-inducing regularization
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing efforts to train pre-trained language models have brought significant improvements to various NLP applications. |
| Approach: | They propose to compress bulky LMs while preserving useful information for a specific task. |
| Outcome: | The proposed method can detach any layer without affecting others, and stretch shallow and wide LMs to be deep and narrow. |